People are utilizing electronic devices for an increasing variety of tasks. With the increase in utilization comes an interest in improving the ways in which users interact with these devices. One approach to enabling a user to provide input to an electronic device involves monitoring a gaze direction with respect to a user device, such that the user can provide input based on where the user is looking. Conventional approaches for determining where a user is looking involve tracking motion of the user's eyes over time. This typically involves an expensive set of electronics positioned in a stationary way in order to provide an acceptable level of accuracy. These approaches, however, are generally not accurate enough to provide for fine control of an element such as a cursor of an interface, particularly for a relatively small portable device with a relatively small display screen. The approaches also tend to be more expensive than is practical for many consumer devices. Further still, small portable devices can be held in a user's hand and thus can be subject to small movements that can negatively impact the accuracy of these approaches.